Portable terminal, hearing aid, and method of indicating positions of sound sources in the portable terminal

ABSTRACT

A portable terminal, a hearing aid, and a method of indicating the position of a sound source in the portable terminal are provided. The method includes receiving sound signals of a plurality of sound sources, determining respective distances between the plurality of sound sources and the portable terminal, and respective directions of the plurality of sound sources from the portable terminal, based on the received sound signals of the plurality of sound sources, determining positions of the plurality of sound sources on the touch screen based on the determined distances and directions of the plurality of sound sources, and displaying the determined positions of the plurality of sound sources on the touch screen.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 15/278,549, filed on Sep. 28, 2016, which a continuation applicationof prior application Ser. No. 14/068,443, filed on Oct. 31, 2013, whichhas issued as U.S. Pat. No. 9,467,786 on Oct. 11, 2016 and was based onand claimed priority under 35 U.S.C. § 119(a) of a Korean patentapplication number 10-2013-0033283, filed on Mar. 28, 2013, in theKorean Intellectual Property Office, the disclosure of which isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The present disclosure relates to a portable terminal, a hearing aid,and a method of indicating the positions of sound sources.

BACKGROUND

Terminals such as smart phones, portable phones, and tablet PersonalComputers (PCs) have recently provided a wider range of services andadditional functions. According to the related art, a terminal isequipped with a camera, a microphone, a speaker, a display, a wirelesscommunication module, and sensors. Accordingly, the terminal providesmany functions in order to satisfy various users' demands.

Particularly, the terminal may provide a variety of Graphical UserInterfaces (GUIs) using the display. If sound sources can be visualizedon the display and controlled, the hearing-impaired as well as ordinaryusers will benefit from the terminal.

For example, if the terminal provides a function of detecting adjacentsound sources and displaying the positions of the sound sources, thehearing-impaired as well as the ordinary users may use the terminal moreconveniently.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a portable terminal which detects adjacentsound sources and indicates the positions of the sound sources, ahearing aid, and a method of indicating the positions of sound sources.

Another aspect of the present disclosure is to provide a portableterminal which enables a user to detect the positions of sound sourcesand selectively hear an intended sound source, a hearing aid, and amethod of indicating the positions of sound sources.

Another aspect of the present disclosure is to provide a portableterminal which enables a user to select the direction, range, distance,or position of an intended sound source by various gestures and outputsa sound of the selected sound source, a hearing aid, and a method ofindicating the positions of sound sources.

In accordance with an aspect of the present disclosure, a method ofindicating the position of a sound source in a portable terminal havinga touch screen is provided. The method includes receiving sound signalsof a plurality of sound sources, determining respective distancesbetween the plurality of sound sources and the portable terminal, andrespective directions of the plurality of sound sources from theportable terminal, based on the received sound signals of the pluralityof sound sources, determining positions of the plurality of soundsources on the touch screen based on the determined distances anddirections of the plurality of sound sources, and displaying thedetermined positions of the plurality of sound sources on the touchscreen.

In accordance with another aspect of the present disclosure, a method ofoutputting a sound in a hearing aid is provided. The method includesreceiving a sound signal of a sound source from the portable terminal,the sound source being selected and having a controlled volume inresponse to a user gesture on a touch screen of the portable terminal,and outputting the received sound signal of the selected sound sourceaccording to the controlled volume.

In accordance with another aspect of the present disclosure, a portableterminal having a touch screen is provided. The portable terminalincludes a touch screen configured to display positions of a pluralityof sound sources, and a controller configured to determine respectivedistances between the plurality of sound sources and the portableterminal, and respective directions of the plurality of sound sourcesfrom the portable terminal, based on the received sound signals of theplurality of sound sources, to determine the positions of the pluralityof sound sources on the touch screen based on the determined distancesand directions of the plurality of sound sources, and to control displayof the determined positions of the plurality of sound sources on thetouch screen.

In accordance with another aspect of the present disclosure, a hearingaid is provided. The hearing aid includes a wireless unit configured toreceive a sound signal of a sound source from the portable terminal, thesound source being selected and having a controlled volume in responseto a user gesture on a touch screen of the portable terminal, and anoutput unit configured to output the received sound signal of theselected sound source according to the controlled volume.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 illustrates operations of a terminal and a sound output deviceaccording to an embodiment of the present disclosure;

FIG. 2 is a block diagram of a terminal according to an embodiment ofthe present disclosure;

FIG. 3 is a flowchart illustrating an operation of displaying at leastone adjacent sound source in a terminal according to an embodiment ofthe present disclosure;

FIG. 4 illustrates a radarscope screen that displays sound sourcesaround a terminal according to an embodiment of the present disclosure;

FIG. 5 illustrates a sound source-position indication screen thatdisplays sound sources around a terminal according to an embodiment ofthe present disclosure;

FIG. 6 is a flowchart illustrating an operation of outputting sounds ofsound sources in a terminal according to an embodiment of the presentdisclosure;

FIGS. 7A and 7B illustrate screens of a terminal that display soundsources around the terminal according to an embodiment of the presentdisclosure;

FIG. 8 is a flowchart illustrating an operation of selecting a soundsource according to a recognized user gesture in a terminal according toan embodiment of the present disclosure;

FIG. 9 illustrates a user gesture input indicating a soundsource-selection direction according to an embodiment of the presentdisclosure;

FIGS. 10A and 10B illustrate a user gesture input indicating a soundsource-selection range according to an embodiment of the presentdisclosure;

FIGS. 11A, 11B, and 11C illustrate a user gesture input indicating asound source-selection distance according to an embodiment of thepresent disclosure;

FIGS. 12A, 12B, 12C, and 12D illustrate user gesture inputs indicating asound source-selection position according to an embodiment of thepresent disclosure;

FIG. 13 is a block diagram of a hearing aid according to an embodimentof the present disclosure; and

FIG. 14 is a flowchart illustrating an operation of a hearing aidaccording to an embodiment of the present disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the embodiments described hereincan be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

By the term “substantially” it is meant that the recited characteristic,parameter, or value need not be achieved exactly, but that deviations orvariations, including for example, tolerances, measurement error,measurement accuracy limitations and other factors known to those ofskill in the art, may occur in amounts that do not preclude the effectthe characteristic was intended to provide.

Various embodiments of the present disclosure will be provided toachieve the above-described technical aspects of the present disclosure.In an implementation, defined entities may have the same names, to whichthe present disclosure is not limited. Thus, various embodiments of thepresent disclosure can be implemented with same or ready modificationsin a system having a similar technical background.

Through the description, ‘user gesture input’ corresponds to an inputthat a user creates by making a gesture on a touch screen (e.g., inorder to control a terminal). For example, user gesture inputs mayinclude a touch, a touch and hold, a drag, a flick, a hold, and thelike.

‘Touch’ corresponds to a gesture of touching a screen with a finger or atouch input unit such as a stylus, a pen, or the like. Specifically, atouch has a preset touch time defined as the difference between atouch-in time when a finger or the touch input unit touches the screenand a touch-out time when the finger or the touch input unit is removedfrom the screen.

‘Touch and hold’ corresponds to a gesture including a touching of ascreen with a finger or the touch input unit and keeping the touch ofthe screen for a predetermined threshold time or longer. The touch andhold has a time difference equal to or larger than a threshold. Forexample, if the difference between a touch-in time and a touch-out timeis greater than or equal to the threshold, then the gesture isdetermined to correspond to a touch and hold. To indicate whether atouch input is touch or touch and hold, if the touch input is kept for athreshold time or longer, a visual or acoustic feedback signal may beprovided to a user.

‘Drag’ corresponds to a gesture of touching a screen with a finger orthe touch input unit and moving the touch to another position of thescreen without detaching the finger or the touch input from the screen.

‘Flick’ corresponds to a gesture of fast dragging a finger or the touchinput unit across a screen. A drag and a flick may be distinguished fromeach other depending on whether the user moves a finger or the touchinput unit at a threshold speed or faster.

‘Hold’ corresponds to a gesture of keeping a touch of a finger or thetouch input unit at a specific point of a screen for a threshold time orlonger without detaching the finger or the touch input unit from thescreen. The hold gesture lasts at a specific point for a threshold timeor longer between a touch-in time and a touch-out time.

FIG. 1 illustrates operations of a terminal and a sound output deviceaccording to an embodiment of the present disclosure.

Referring to FIG. 1, a terminal 100 may be a portable terminal. Theterminal 100 may locate one or more sound sources around the terminal100 (or a user) and display a terminal image 10 representing theterminal 100 and sound source images 11 to 17 representing the soundsources at the corresponding positions of the sound sources with respectto the terminal 100.

A sound source refers to a source or its equivalent from which a soundis emitted. For example, sound sources may include a person that speaks,a player that performs music, a speaker that outputs music, a phone thatrings, and the like. Besides, any other source that outputs a sound maybe a sound source.

The terminal 100 may select the sound source image of a sound source toemit a sound according to a user gesture input from among the soundsource images 11 to 17, and may output a sound of the selected soundsource. In addition, the terminal 100 may provide the sound of theselected sound source to a sound output device 300 (e.g., a hearing aid,an ear-microphone, a headset, or the like).

The terminal 100 may select at least one volume level for a sound sourcefrom among a plurality of preset volume levels according to a usergesture input and output a sound of the sound source at the selectedvolume level. The terminal 100 may also output the sound of the selectedsound source at the selected volume level to the sound output device300.

The sound output device 300 may be a device that outputs a sound, suchas a hearing aid, an ear-microphone, a headset, or the like. Variousembodiments of the present disclosure will be described in the contextof a hearing aid being used as the sound output device 300, by way ofexample. The hearing aid 300 may receive sound signals of one or moresound sources through microphones, amplify the sound signals, and outputthe amplified sound signals audibly to a user through a speaker.

The hearing aid 300 may communicate with the terminal 100 wirelessly orwiredly and may receive a sound of a sound source selected from amongone or more sound sources from the terminal 100 and output the receivedsound. The hearing aid 300 may receive a sound of a sound source with avolume controlled to a volume level selected by the terminal 100 andoutput the received volume-controlled sound.

FIG. 2 is a block diagram of the terminal according to an embodiment ofthe present disclosure.

Referring to FIG. 2, the terminal 100 may include a reflected-wavetransceiver 103, a sound source-distance calculator 105, a soundsource-direction calculator 107, and a controller 110. The terminal 100may further include a mobile communication module 120, asub-communication module 130, a multimedia module 140, a camera module150, a Global Positioning Service (GPS) module 157, an Input/Output(I/O) module 160, a sensor module 170, a memory 175, and a power supply180. Additionally, the terminal 100 may include at least one touchscreen 190 and at least one touch screen controller 195.

The controller 110 may include a Central Processing Unit (CPU) 111, aRead Only Memory (ROM) 112 that stores a control program to control theterminal 100, and a Random Access Memory (RAM) 113 that stores anexternal input signal or data or that is used as a memory space for anoperation of the terminal 100. The CPU 111 may include a various numberof cores. For example, the CPU 111 may include a single core, dualcores, triple cores, quadruple cores, or the like. The CPU 111, the ROM112, and the RAM 113 may be connected to one another through an internalbus. The controller 110 may control the reflected-wave transceiver 103,the sound-source distance calculator 105, the sound-source directioncalculator 107, the mobile communication module 120, thesub-communication module 130, the multimedia module 140, the cameramodule 150, the GPS module 157, the I/O module 160, the sensor module170, the memory 175, the power supply 180, the touch screen 190, and thetouch screen controller 195.

The reflected-wave transceiver 103 may transmit an electromagnetic-wavesignal in all directions and receive reflected-wave signals resultingfrom reflecting the electromagnetic-wave signal from objects under thecontrol of the controller 110. The controller 110 may use thereflected-wave signals received from the reflected-wave transceiver 103in detecting the objects around the terminal 100.

The sound source-distance calculator 105 receives a voice or sound fromsound sources through first to nth microphones 162-1 to 162-n,identifies one or more sound sources, and calculates the distancebetween the terminal 100 and each of the sound sources. Specifically,the sound source-distance calculator 105 generates a plurality ofmicrophone output signals by combining voices or sounds received fromsound sources through at least two microphones among the first to nthmicrophones 162-1 to 162-n and calculates the distance to each soundsource based on the sound attenuation rates of the plurality ofmicrophone output signals with respect to distances.

The sound source-direction calculator 107 receives voices or sounds fromsound sources through the first to nth microphones 162-1 to 162-n,identifies one or more sound sources, and calculates the direction ofeach sound source under the control of the controller 110. Specifically,the sound source-direction calculator 107 calculates the direction ofeach sound source based on a time delay between voices or soundsreceived from sound sources through at least two microphones among thefirst to nth microphones 162-1 to 162-n. Information about the directionof each sound source is provided to the controller 110.

The controller 110 may locate at least one object around the terminal100 (or the user) using the reflected waves received from thereflected-wave transceiver 103.

The controller 110 may identify at least one sound source by receivingvoices or sounds from sound sources through the first to nth microphones162-1 to 162-n. Then the controller 110 analyses the frequency componentof the at least one sound source and compares the analyzed frequencycomponent with a predetermined frequency component, thereby determiningthe type of the at least one sound source. For example, the controller110 may determine whether a voice or sound from the at least one sourcesound is a person's voice, a sound coming from an animal, a phone ring,a sound from a vehicle, or the like.

The controller 110 may also calculate the strength of a sound from eachsound source, receive information about the distance from the terminal100 to each sound source from the sound source-distance calculator 105,and receive information about the direction of each sound source fromthe sound source-direction calculator 107.

According to various embodiments of the present disclosure, thecontroller 110 may receive from the hearing aid 300 voices or sounds ofsound sources received through a plurality of microphones of the hearingaid 300, instead of the first to nth microphones 162-1 to 162-n.Thereafter, the controller 110 may identify at least one sound source,and calculate the sound strength, direction, and distance to theterminal 100 of each sound source. For example, the hearing aid 300 mayinclude at least two microphones and transmit voices or sounds of soundresources received through the at least two microphones to the terminal100 by communication.

The controller 110 controls display of a terminal image representing theterminal 100 and sound source images representing sound sources on thetouch screen 190 based on the positions of the detected objects, thestrengths of sounds from the sound sources, the directions of the soundsources, and the distances between the sound sources and the terminal100. For example, the controller 110 may display the terminal image andthe sound source images representing the detected sound sources on aradarscope screen along with at least one of the distance between theterminal 100 and each sound source, the azimuth angle of each soundsource, and the strength of a sound from each sound source.

If a sound source is a person's voice, the controller 110 may detect theposition of the person, receive information of the person correspondingto the sound source from an external entity like a communication serviceprovider, and display the sound source along with the information of theperson corresponding to the sound source. For example, the controller110 may display the phone number, name, and/or the like of the personcorresponding to the sound source.

The distances and directions of sound sources may be displayed withrespect to the terminal 100 on the radarscope screen. The sound sourceimages may take the form of figures, characters, or the like. Dependingon the sound strengths and distances of sound sources, sound sourceimages representing the sound sources may be different in color or size.

For example, if a sound source emits a loud sound, a sound source imagerepresenting the sound source may be displayed larger or in a deepercolor than a sound source image representing a sound source that emits aweak sound. If a sound source is near to the terminal 100, a soundsource image representing the sound source may be displayed larger or ina deeper color than a sound source image representing a remote soundsource. The shape, size, and color of the sound source image of a soundsource may be changed selectively according to the sound strength anddistance of the sound source.

Alternatively, the controller 110 may control display of the terminalimage and the sound source images on a sound source-position indicationscreen having a predetermined background, instead of the radarscopescreen.

As an example, while a terminal image and at least one sound sourceimage are being displayed on the touch screen 190, the controller 110monitors reception of a user gesture input made by a finger a touchinput unit 168 such as an electronic pen. When the user makes a gestureover the touch screen with a finger or the input unit 168, thecontroller 110 may detect the resulting user gesture input. In anembodiment of the present disclosure, the user gesture input may be atleast one of a touch, a touch and hold, a flick, and a hold. Besides,many other user gesture inputs are available.

The controller 110 may select the sound image of a sound source thatwill output a sound according to a user gesture input from among one ormore sound source images and may control output of a sound from theselected sound source through a speaker 163.

Upon selection of at least one of a plurality of predetermined volumelevels for a sound source image in response to a user gesture input, thecontroller 110 may control the speaker 163 to output a sound of a soundsource corresponding to the sound source image at the selected volumelevel.

According to various embodiments of the present disclosure, thecontroller 110 may select the sound source image of a sound source thatwill output a sound from among one or more sound source images accordingto a user gesture input and may control transmission of a sound of theselected sound source to the hearing aid 300 so that the hearing aid 300may output the sound of the selected sound source.

Upon selection of at least one of a plurality of predetermined volumelevels for a sound source image in response to a user gesture input, thecontroller 110 may control transmission of a sound of the sound sourcecorresponding to the sound source image with a volume controlled basedon the selected volume level to the hearing aid 300 so that the hearingaid 300 may output the volume-controlled sound of the selected soundsource.

The mobile communication module 120 connects the terminal 100 to anexternal device through one or more antennas (not shown) by mobilecommunication under the control of the controller 110. The mobilecommunication module 120 transmits wireless signals to or receiveswireless signals from a portable phone (not shown), a smart phone (notshown), a tablet PC (not shown), or other devices (not shown) that havephone numbers to be input to the terminal 100, for voice call, videocall, Short Message Service (SMS), or Multimedia Messaging Service(MIMS).

The sub-communication module 130 may include at least one of a WirelessLocal Area Network (WLAN) module 131 and a short-range communicationmodule 132. For example, the sub-communication module 130 may includethe WLAN module 131, or the short-range communication module 132,individually or in combination.

The WLAN module 131 may be connected to the Internet under the controlof the controller 110 in a place at which a wireless Access Point (AP)(not shown) is installed. The WLAN module 131 supports the WLANstandard, the Institute of Electrical and Electronics Engineers (IEEE)802.11x. The short-range communication module 132 may conductshort-range wireless communication between the terminal 100 and an imageforming device (not shown) under the control of the controller 110. Theshort-range communication may conform to Bluetooth, Infrared DataAssociation (IrDA), Wireless Fidelity (Wi-Fi) Direct, Near FieldCommunication (NFC), and the like.

The terminal 100 may communicate with the hearing aid 300 through atleast one of the mobile communication module 120 and thesub-communication module 130. In addition, the terminal 100 may transmita sound of a sound source to the hearing aid 300 through at least one ofthe mobile communication module 120 and the sub-communication module130. For example, the terminal 100 may transmit a sound of auser-selected sound source to the hearing aid 300 through at least oneof the mobile communication module 120 and the sub-communication module130. The terminal 100 may also transmit a sound of a sound source with avolume controlled based on a user-selected volume level to the hearingaid 300 through at least one of the mobile communication module 120 andthe sub-communication module 130.

The terminal 100 may include at least one of the mobile communicationmodule 120, the WLAN module 131, and the short-range communicationmodule 132 according to the capabilities of the terminal 100. Forexample, the terminal 100 may include a combination of the mobilecommunication module 120, the WLAN module 131, and the short-rangecommunication module 132 according to the capabilities of the terminal100. In the present disclosure, one or more of the mobile communicationmodule 120, the WLAN module 131, and the short-range communicationmodule 132 are collectively referred to as a transceiver, which shouldnot be construed as limiting the scope of the present disclosure.

The multimedia module 140 may include a broadcasting communicationmodule 141, an audio play module 142, or a video play module 143. Thebroadcasting communication module 141 may receive a broadcast signal(e.g., a TV broadcast signal, a radio broadcast signal, a data broadcastsignal, or the like) and additional broadcasting information (e.g., anElectronic Program Guide (EPG), Electronic Service Guide (ESG), or thelike) from a broadcasting station through a broadcasting communicationantenna (not shown) under the control of the controller 110. The audioplay module 142 may open a stored or received digital audio file (e.g.,a file having such an extension as mp3, wma, ogg, way, or the like)under the control of the controller 110. The video play module 143 mayopen a stored or received digital video file (e.g., a file having suchan extension as mpeg, mpg, mp4, avi, mov, mkv, or the like) under thecontrol of the controller 110. The video play module 143 may also open adigital audio file.

According to various embodiments of the present disclosure, themultimedia module 140 may include any combination of the broadcastingcommunication module 141, the audio play module 142, and the video playmodule 143. For example, the multimedia module 140 may include the audioplay module 142 and the video play module 143 except the broadcastingcommunication module 141.

According to various embodiments of the present disclosure, the audioplay module 142 or the video play module 143 of the multimedia module140 may be incorporated into the controller 110.

The camera module 150 may include at least one of a first camera 151 anda second camera 152, to capture a still image or a video under thecontrol of the controller 110. Further, the first camera 151 or thesecond camera 152 may include at least one of a barrel 155 to zoom in orzoom out relative to an object when the object is captured, a motor 154to control movement of the barrel 155, and a flash 153 to provide anauxiliary light source for capturing an image. The first camera 151 maybe disposed on the front surface of the terminal 100, and the secondcamera 152 may be disposed on the rear surface of the terminal 100.Alternatively, the first camera 151 and the second camera 152 may bearranged near to each other (e.g., the distance between the first camera151 and the second camera 152 may be between 1 cm and 8 cm) in order tocapture a three-dimensional still image or video. The camera module 150transmits a captured picture or image to the controller 110.

Each of the first and second cameras 151 and 152 may include a lenssystem and an image sensor. Each of the first and second cameras 151 and152 may convert an optical signal received (or captured) at the lenssystem to an electrical image signal and output the electrical imagesignal to the controller 110. Therefore, the user may capture a video ora still image using the first and second cameras 151 and 152.

The GPS module 157 may receive signal waves from a plurality of GPSsatellites (not shown) in Earth orbit and calculate the position of theterminal 100 based on the Time of Arrivals (ToAs) of satellite signalsfrom the GPS satellites to the terminal 100.

The I/O module 160 may include at least one of a plurality of buttons161, the first to nth microphones 162-1 to 162-n, the speaker 163, avibration motor 164, a connector 165, a keypad 166, an earphoneconnector jack 167, an input unit 168, and the like. However, the I/Omodule 160 is not limited to these components and may provide cursorcontrol to communicate with the controller 110 and to control cursormovement on the touch screen 190 through a mouse, a track ball, a joystick, cursor directional keys, or the like.

The buttons 161 may be formed on the front surface, a side surface, orthe rear surface of a housing of the terminal 100, and may include atleast one of a power/lock button (not shown), a volume button (notshown), a menu button, a home button, a back button, a search button(not shown), and the like.

The first to nth microphones 162-1 to 162-n receive voices or soundsfrom sound sources and convert the received voices or sounds toelectrical signals under the control of the controller 110. For example,the first to nth microphones 162-1 to 162-n may receive voices or soundsfrom sound sources around the terminal 100, such as music played by aplayer, speech from a person, a phone ringing sound, a speech from aspeaker, and the like.

The speaker 163 may output sounds corresponding to various signals(e.g., a wireless signal, a broadcast signal, a digital audio file, adigital video file, a photo shot, and the like) received from the mobilecommunication module 120, the sub-communication module 130, themultimedia module 140, and the camera module 150 to the outside of theterminal 100. According to various embodiments of the presentdisclosure, the speaker 163 may amplify a sound of a sound sourceselected from among sound sources around the terminal 100 by the userand output the amplified sound to the outside under the control of thecontroller 110. The volume of a sound of a sound source may becontrolled according to a user gesture input. One or more speakers 163may be disposed at an appropriate position or positions of the housingof the terminal 100.

The vibration motor 164 may convert an electrical signal to a mechanicalvibration under the control of the controller 110. For example, when theterminal 100 receives an incoming voice call from another device (notshown) in vibration mode, the vibration motor 164 operates. One or morevibration motors 164 may be mounted inside the housing of the terminal100. The vibration motor 164 may operate in response to a user's touchon the touch screen 190 and a continuous movement of the touch on thetouch screen 190.

The connector 165 may be used as an interface to connect the terminal100 to an external device (not shown) or a power source (not shown). Theterminal 100 may transmit data stored in the memory 175 to the externaldevice via a cable connected to the connector 165 or may receive datafrom the external device via the cable, under the control of thecontroller 110. The terminal 100 may receive power or charge a battery(not shown) from the power source via the cable connected to theconnector 165.

The keypad 166 may receive a key input from the user to control theterminal 100. The keypad 166 includes a physical keypad (not shown)formed in the terminal 100 or a virtual keypad (not shown) displayed onthe touch screen 190. According to various embodiments of the presentdisclosure, the physical keypad may not be provided according to thecapabilities or configuration of the terminal 100.

An earphone (not shown) may be inserted into the earphone connector jack167 and thus connected to the terminal 100.

The input unit 168 may be inserted into the terminal 100. When the inputunit 168 is used, the input unit 168 may be extended or detached fromthe terminal 100. An insertion and removal detecting switch (not shown)is provided at an area of the terminal 100 in which the input unit 168is inserted, in order to operate according to insertion and removal ofthe input unit 168 and provide a signal indicating insertion or removalof the input unit 168 to the controller 110. The insertion and removalswitch is configured to directly or indirectly contact the input unit168 in the insertion area of the input unit 168, when the input unit 168is inserted. Thus, the insertion and removal switch generates a signalindicating insertion or removal of the input unit 168 and provides thegenerated signal to the controller 110.

The sensor module 170 includes at least one sensor to detect a state ofthe terminal 100. For example, the sensor module 170 may include aproximity sensor to detect whether the user is close to the terminal100, an illuminance sensor (not shown) to detect the amount of ambientlight around the terminal 100, a motion sensor (not shown) to detect amotion of the terminal 100 (for example, rotation, acceleration, orvibration of the terminal 100), a geomagnetic sensor 172 to detect apoint of the compass using the earth's magnetic field, a gravity sensor(not shown) to detect the direction of gravity, and an altimeter (notshown) to detect an altitude by measuring the air pressure. At least onesensor may detect a state of the terminal 100, generate a signalcorresponding to the detected state, and transmit the generated signalto the controller 110. A sensor may be added to or removed from thesensor module 170 according to the capabilities of the terminal 100.

The memory 175 may store input/output signals or data in accordance withoperations of the mobile communication module 120, the sub-communicationmodule 130, the multimedia module 140, the camera module 150, the GPSmodule 157, the I/O module 160, the sensor module 170, and the touchscreen 190 under the control of the controller 110. The memory 175 maystore a plurality of programs to control the terminal 100 or thecontroller 110, and a plurality of applications.

The term “memory” covers the memory 175, the ROM 112 and the RAM 113within the controller 110, or a memory card (not shown) (e.g., a SecureDigital (SD) card, a memory stick, and the like) mounted to the terminal100. The memory may include a non-volatile memory, a volatile memory, aHard Disk Drive (HDD), or a Solid State Drive (SSD).

The memory 175 may store applications having various functions such asnavigation, video call, game, and time-based alarm applications, imagesused to provide GUIs related to the applications, user information,text, databases or data related to a method of processing a touch input,background images (e.g., a menu screen, a waiting screen, a radarscopescreen, a sound source-position indication screen, and the like) oroperation programs required to operate the terminal 100, and stillimages and pictures captured by the camera module 150. The memory 175 isa non-transitory machine-readable medium (e.g. a non-transitorycomputer-readable medium). A non-transitory machine-readable medium maybe defined as a medium that provides data to a machine so that themachine may perform a specific function. The non-transitorymachine-readable medium may be a storage medium. The memory 175 mayinclude a volatile medium and a non-volatile medium. The media should bea type of media that transfers commands detectable by a physical devicethat reads the commands to the machine.

According to various embodiments of the present disclosure,non-transitory machine-readable media include at least one of a floppydisk, a flexible disk, a hard disk, a magnetic tape, a Compact Disk ReadOnly Memory (CD-ROM), an optical disk, a punch card, a paper tape, aRAM, a Programmable ROM (PROM), an Erasable PROM (EPROM), a Flash-EPRIM,and the like.

The power supply 180 may supply power to one or more batteries (notshown) mounted in the housing of the terminal 100 under the control ofthe controller 110. The one or more batteries supply power to theterminal 100. Further, the power supply 180 may supply power receivedfrom an external power source (not shown) via the cable connected to theconnector 165 to the terminal 100. The power supply 180 may also supplypower received wirelessly from the external power source to the terminal100 by a wireless charging technology.

According to various embodiments of the present disclosure, the terminal100 may include at least one touch screen to provide User Interfaces(UIs) corresponding to various services (e.g., call, data transmission,broadcasting, photo shot, and sound source-position indication). Eachtouch screen may transmit an analog signal corresponding to at least onetouch on a UI to a touch screen controller corresponding to the touchscreen. As many touch screen controllers as touch screens may beprovided to receive analog signals from the touch screens. According tovarious embodiments of the present disclosure, the terminal 100 mayinclude at least one touch screen. For the convenience of description, acase of using a single touch screen will be described.

The touch screen 190 may receive at least one user gesture input througha user's body (e.g., fingers including an index finger) or the touchinput unit 168 (e.g., a stylus pen, an electronic pen, or the like). Thetouch screen 190 includes a pen recognition panel (not shown) torecognize a user gesture input from the stylus pen or the electronicpen. The pen recognition panel may recognize the distance between thepen and the touch screen 190 by a magnetic field. In addition, the touchscreen 190 may receive a continuous movement of at least one successivetouch as a user gesture input. According to various embodiments of thepresent disclosure, the touch screen 190 may provide an analog signalcorresponding to a user gesture input to the touch screen controller195.

The touch screen 190 may be configured as, for example, a resistivetype, a capacitive type, an infrared type, an acoustic wave type, and/orthe like.

To sequentially or simultaneously receive gesture inputs from the user'sbody or the touch input unit 168, the touch screen 190 may include atleast two touch screen panels (not shown) to respectively detect touchesor proximity of the user's body and the touch input unit 168. The atleast two touch screen panels provide different output values to thetouch screen controller 195. The touch screen controller 195distinguishes the values received from the at least two touch screenpanels from each other, and determines whether an input value from thetouch screen 190 has been generated from a gesture input of the user'sbody or from the touch input unit 168.

More specifically, the touch screen 190 may include a stack of a panelto detect an input from a finger or the input unit 168 by a variation ininduced electrical power, and a panel to detect a touch from the fingeror the input unit 168. The two panels are spaced partially from eachother or brought into contact. The touch screen 190 includes a largenumber of pixels and displays an image at the pixels. According tovarious embodiments of the present disclosure, the touch screen 190 maybe configured as a Liquid Crystal Display (LCD), an Organic LightEmitting Diode (OLED) display, an LED display, or the like.

FIG. 3 is a flowchart illustrating an operation of displaying at leastone sound source around the terminal 100 according to an embodiment ofthe present disclosure.

Referring to FIG. 3, at operation 310, the terminal 100 may determinethe bearings of East, West, South, and North with respect to theposition of the terminal 100 by means of the geomagnetic sensor 172.Specifically, the terminal 100 may detect the position of the terminal100 by means of the GPS module 157 and may determine the bearings ofEast, West, South, and North with respect to the position of theterminal 100 by means of the geomagnetic sensor 172.

At operation 320, the terminal 100 may detect objects around theterminal 100 using reflected waves and locate the detected objects. Forexample, the terminal 100 may transmit an electromagnetic-wave signal inall directions through the reflected-wave transceiver 113 and receivereflected-wave signals corresponding to the transmittedelectromagnetic-wave signals being reflected from objects. The terminal100 may detect one or more objects around the position of the terminal100 and may locate the one or more objects using the receivedreflected-wave signals.

At operation 330, the terminal 100 may distinguish one or more soundsources from one another by analyzing sounds received through aplurality of microphones. For example, the terminal 100 may distinguishone or more sound sources by analyzing voices or sounds received fromthe sound sources through the first to nth microphones 162-1 to 162-n.According to various embodiments of the present disclosure, the terminal100 may receive from the hearing aid 300 voices or sounds that thehearing aid 300 has received from sound sources through a plurality ofmicrophones and may distinguish one or more sound sources from oneanother. For example, rather than receiving the voices or sounds throughthe first to nth microphones 162-1 to 162-n, the terminal 100 mayreceive voices or sounds that the hearing aid 300 has received fromsound sources through a plurality of microphones included in the hearingaid 300. Thereafter, the terminal 100 may distinguish among the one ormore sound sources from which the voices or sounds originate (or areemitted).

At operation 340, the terminal 100 calculates the sound strength,direction, and distance of each of the distinguished sound sources. Forexample, the terminal 100 may calculate the sound strength of each soundsource by analyzing voices or sounds of the sound sources through thefirst to nth microphones 162-1 to 162-n. In addition, the controller 110may calculate the distance between the terminal 100 and each soundsource through the sound source-distance calculator 103 and maycalculate the direction of each sound source through the soundsource-direction calculator 105.

At operation 350, the terminal 100 displays the terminal 100 and one ormore sound sources around the terminal 100 based on the positions of thedetected objects, the calculated sound strengths, directions, anddistances of the sound sources. For example, the terminal 100 maydisplay the relative positions of the one or more sound sources aroundthe terminal 100.

For example, the terminal 100 selects at least one sound source to bedisplayed on the touch screen 190 from among one or more sound sourcesaround the terminal 100 and determines a position at which the selectedsound source is to be displayed on the touch screen 190. Thereafter, theterminal 100 displays the at least one sound source at the determinedposition of the touch screen 190.

Specifically, the terminal 100 may select at least one sound sourcewithin a predetermined distance from the terminal 100 or at least onesound source having a sound strength equal to or larger than apredetermined (or selected) level from among one or more sound sourcesaround the terminal 100, so that the selected at least one sound sourcemay be displayed on the touch screen 190.

The display position of the selected at least one sound source may bedetermined, taking into account the size of the touch screen 190 and thedistance between the terminal 100 and the at least one sound source.

For example, the center point of the touch screen 190 may be set as theposition of the terminal 100 and the display position of the at leastone sound source may be determined with respect to the center point ofthe touch screen 190 based on the distance between the terminal 100 andthe at least one sound source, taking into account the size of the touchscreen 190. For example, the terminal 100 may display the position ofthe terminal and the relative positions of the at least one sound sourcebased on a scaling according to the size of the touch screen 190.

The terminal 100 may display a terminal image representing the terminal100 at the position of the terminal 100 on the touch screen 190 and atleast one sound source image representing the at least one sound sourceat the determined position of the at least one sound source on the touchscreen 190.

The terminal 100 may display the terminal image and the at least onesound source image on a predetermined type of screen and mayadditionally display at least one of the distance, an azimuth angle, anda type of the at least one sound source on the screen.

A sound source image may take the form of a figure, a character, or thelike to represent a sound source. The sound source image may bedisplayed in a different color or size according to the sound strengthand distance of the sound source. For example, if a sound source emits aloud sound, a sound source image representing the sound source may bedisplayed larger or in a deeper color than a sound source imagerepresenting a sound source that emits a weak sound. As another example,if a sound source is near to the terminal 100, a sound source imagerepresenting the sound source may be displayed larger or in a deepercolor than a sound source image representing a remote sound source. Theshape, size, and color of a sound source image corresponding to a soundsource may be changed selectively according to the sound strength anddistance of the sound source. A sound source image may be formed into ashape matching the type of a sound source. For example, the sound sourceimage may be formed into a shape representing that the type of the soundsource corresponds to a person, a phone, or an instrument, and the like.

If the type of a sound source is a person's voice, the terminal 100 maydetect the position of the person, receive information (e.g., the phonenumber and name) of the person corresponding to the sound source from anexternal entity like a communication service provider, and display thesound source along with information of the corresponding to the soundsource. According to various embodiments of the present disclosure, theterminal 100 may determine that the person from whom the sound isreceived corresponds to a contact stored in the terminal 100. As aresult, the terminal 100 may retrieve and display contact information(e.g., phone number, name, and/or the like) associated with the person.

The terminal 100 and sound sources around the terminal 100 may bedisplayed in a predetermined indication scheme such as a radarscopedisplay scheme, a sound source-position indication scheme other than theradarscope display scheme, or other unmentioned schemes.

An embodiment of the present disclosure in which the terminal 100 andsound sources around the terminal 100 are displayed in the radarscopedisplay scheme will first be described.

FIG. 4 illustrates a radarscope screen that displays sound sourcesaround a terminal according to an embodiment of the present disclosure.

Referring to FIG. 4, the distances and directions of sound sources maybe displayed with respect to the terminal 100 on a radarscope screen.The bearings of North, West, South, and East are marked with N, W, S andE on the radarscope screen. A terminal image 40 representing theterminal 100 and one or more sound source images 41 to 44 respectivelyrepresenting sound sources may be displayed on the radarscope screen.The radarscope screen may further display the distance between theterminal 100 and each sound source, and the sound strength 47 andazimuth angle 49 of the sound source. For example, if a source image ofsource images 41 to 44 is selected, the terminal 100 may displaydetailed information about the sound source and/or the correspondingsound. For example, the terminal 100 may display the distance betweenthe terminal and the selected sound source, the sound strength of thesound from the selected sound source, contact information or otherinformation relating to characteristics of the selected sound source,and the like.

FIG. 5 illustrates a screen that displays the positions of sound sourcesaround a terminal according to an embodiment of the present disclosure.

Referring to FIG. 5, a sound source-position indication screen maydisplay a terminal image 50 representing the terminal 100 and one ormore sound source images 51 to 55 representing sound sources. If a soundsource outputs a loud sound, the sound source image of the sound sourcemay be displayed in a deeper color than the sound source image of asound source that outputs a weak sound. The sound source image of asound source may indicate whether the type of the sound source is aperson's voice (Bla Bla) or a phone ring. According to variousembodiments of the present disclosure, the sound source image of thesound source may indicate other types (e.g., common types) of soundsources.

With sound sources around the terminal 100 displayed on a screen in theabove manner, the terminal 100 may select a sound source according to auser gesture input, output a sound of the selected sound source, andcontrol the volume of the output sound.

FIG. 6 is a flowchart illustrating an operation of outputting sounds ofsound sources in a terminal according to an embodiment of the presentdisclosure. FIGS. 7A and 7B illustrate screens of a terminal thatdisplay sound sources around the terminal according to an embodiment ofthe present disclosure.

Referring to FIGS. 6, 7A and 7B, at operation 610, the terminal 100 maydetermine whether a user has touched a sound source image on a screenthat displays a terminal image 70 and sound source images 71 to 75representing sound sources around the terminal 100. The user may touch asound source image on the screen with a finger or the input unit 168such as a touch pen or a stylus pen.

If the terminal 100 determines that the user has not touched a soundsource image on the screen at operation 610, then the terminal 100 maycontinue to poll for a user touch on a sound source image (e.g., untilthe terminal 100 is interrupted to perform another correspondingfunction based on user input or other operations).

In contrast, if the terminal 100 determines that the user has touched asound source image on the screen at operation 610, then upon detectingof a touch on a sound source image, the terminal proceeds to operation620 at which the terminal 100 may output a sound of a sound sourcecorresponding to the touched sound source image. For example, when thesound source image 71 corresponding Bla Bla 3 is touched as illustratedin FIG. 7A, the terminal 100 may output a sound of a sound sourcecorresponding to the touched sound source image 71.

According to various embodiments of the present disclosure, the terminal100 may transmit a sound of the sound source corresponding to thetouched sound source image to the hearing aid 300. For example, theterminal 100 may transmit the sound of the sound source corresponding tothe touched (e.g., selected) sound source image to the hearing aid 300so that the hearing aid 300 may output the sound of the sound source.

When the sound source image 71 corresponding Bla Bla 3 is touched asillustrated in FIG. 7A, the touched sound source image 71 may bedisplayed in a different manner from the other sound source images 72 to75 to indicate the touched sound source image 71 as illustrated in FIG.7B. For example, the touched sound source image 71 may be displayed soas to have a solid outline, and the other sound source images 72 to 75may be displayed so as to have a dotted outline. According to variousembodiments of the present disclosure, other indicia may be used toidentify the touched sound source image (e.g., shading, bolding, and thelike).

According to various embodiments of the present disclosure, if aplurality of sound source images are touched, sounds of sound sourcescorresponding to touched (e.g., selected) the plurality of sound sourceimages may be output and the touched sound source images may bedisplayed differently from the other sound source images so that thetouched sound source images may be identified.

At operation 630, the terminal 100 determines whether the user hasselected a volume level. For example, one of an intense level, astandard level, and a mute level may be selected according to a usergesture input. The intense level may be a highest volume level, thestandard level may be a medium volume level, and the mute level may be amute sound. The volume levels may further include a level that increasesthe volume of a selected sound source above those of the other soundsources or a level that decreases the volume of a selected sound sourcebelow those of the other sound sources.

According to various embodiments of the present disclosure, a variousnumber of volume levels may be defined (e.g., besides the three volumelevels). According to various embodiments of the present disclosure, thevolume level may be set by setting a volume bar between a muted positionand a position corresponding to a maximum volume. The volumes of theintense level, the standard level, and the mute level may be set by auser or a developer.

If the terminal 100 determines that the user has selected the intenselevel, then the terminal 100 may proceed to operation 632 at which theterminal 100 may output the sound of the sound source with a volumecorresponding to the intense level.

If the terminal 100 determines that the user has selected the standardlevel, then the terminal 100 may proceed to operation 634 at which theterminal 100 may output the sound of the sound source with a volumecorresponding to the standard level.

If the terminal 100 determines that the user has selected the mutelevel, then the terminal 100 may proceed to operation 636 at which theterminal 100 may mute the sound of the sound source.

According to various embodiments of the present disclosure, the terminal100 may transmit the sound of the sound source with a volume controlledto a selected volume level to the hearing aid 300 so that the hearingaid 300 may output the volume-controlled sound.

In contrast, if the terminal 100 determines that no volume level hasbeen selected during outputting the sound of the sound source atoperation 630, then the terminal 100 proceeds to operation 640 at whichthe terminal 100 monitors a touch on the sound source imagecorresponding to the sound source that is outputting sound. For example,at operation 640, the terminal 100 determines whether the user touchesthe sound source image corresponding to the sound source outputtingsound.

If the terminal 100 determines that the sound source image correspondingto the sound source outputting sound is touched, then the terminal 100proceeds to operation 650 at which the terminal 100 discontinuesoutputting the sound of the sound source.

If the terminal 100 determines that the sound source image correspondingto the sound source outputting sound is not touched, then the terminal100 may proceed to operation 630.

According to various embodiments of the present disclosure, if theterminal 100 is outputting the sound of the sound source to the hearingaid 300, the terminal 100 may discontinue outputting the sound of thesound source to the hearing aid 300.

As described above, with sound sources around the terminal 100 displayedon a screen, a user may select a sound source on the screen and theterminal 100 or the hearing aid 300 may output a sound of the selectedsound source. In addition, the user may adjust the volume of the soundof the sound source. Therefore, a hearing-impaired person may hear anintended sound source.

While it has been described in the above embodiment of the presentdisclosure that the terminal 100 selects a sound source from which soundis to be output if a user gesture input is a touch, the terminal 100 mayselect a user-intended sound source by recognizing many user gestureinputs other than a touch.

FIG. 8 is a flowchart illustrating an operation of selecting a soundsource according to a recognized user gesture in a terminal according toan embodiment of the present disclosure.

Referring to FIG. 8, at operation 810, the terminal 100 recognizes(e.g., detects) a user gesture. The user gesture input may be a singlegesture input or successive inputs of a plurality of user gestures.

For example, the user gesture input may be a drag input. The usergesture input may be a flick input following a touch and hold input. Theuser gesture input may be a flick and then hold input after a touch andhold input. The user gesture input may be a hold input at a specificpoint of the circumference of a circle having a radius defined by aflick input following a touch and hold input. According to variousembodiments of the present disclosure, the user gesture may be a voiceinput.

At operation 820, the terminal 100 determines whether the user gestureinput indicates a sound source-selection direction. For example, a usergesture input indicating a sound source-selection direction may be adrag input in an intended sound-output direction after a terminal imageis selected.

FIG. 9 illustrates a user gesture input indicating a soundsource-selection direction according to an embodiment of the presentdisclosure.

Referring to FIG. 9, while sound source images 91 to 95 representing oneor more sound sources around the terminal 100 are being displayed on ascreen, upon receipt of a drag input from a terminal image 90 toward thesound source image 92 corresponding to Bla Bla 3, the terminal 100determines a sound source-selection direction based on the drag input.

If the terminal 100 determines that the user gesture input indicates asound source-selection direction at operation 820, then upon the soundsource-selection direction being determined, the terminal 100 proceedsto operation 822 at which the terminal 100 selects a sound source in thesound source-selection direction and outputs a sound of the selectedsound source.

In contrast, if the terminal 100 determines that the user gesture inputdoes not indicate a sound source-selection direction at operation 820,then the terminal 100 proceeds to operation 830 at which the terminal100 determines whether the user gesture input indicates a soundsource-selection range. For example, a user gesture input indicating asound source-selection range may be a flick input to a specific rangeafter a touch and hold input to a terminal image.

FIGS. 10A and 10B illustrate a user gesture input indicating a soundsource-selection range according to an embodiment of the presentdisclosure.

Referring to FIGS. 10A and 10B, while a terminal image 1000 and soundsource images 1001 to 1005 representing one or more sound sources aroundthe terminal 100 are being displayed on a screen, upon receipt of atouch and hold input to the terminal image 1000 as illustrated in FIG.10A, and then a flick input corresponding to a specific range (e.g., a5-m range) as illustrated in FIG. 10B, the terminal 100 determines asound source-selection range according to the received inputs. Theterminal 100 may display a touched and held point according to the touchand hold input and then determine a specific range according to thelength of the flick input. According to various embodiments of thepresent disclosure, the source-selection range may be configured usinganother input or through a menu selection.

If the terminal 100 determines that the user gesture input indicates asound-source selection range at operation 830, then the terminal 100proceeds to operation 832 at which the terminal 100 selects at least onesound source within the indicated range and outputs the sound of theselected at least one sound source.

In contrast, if the terminal 100 determines that the user gesture inputdoes not indicate a sound-source selection range at operation 830, thenthe terminal 100 proceeds to operation 840 at which the terminal 100determines whether the user gesture input indicates a soundsource-selection distance. For example, a user gesture input indicatinga sound source-selection distance may be a flick input and then a holdinput after a touch and hold input.

FIGS. 11A to 11C illustrate a user gesture input indicating a soundsource-selection distance according to an embodiment of the presentdisclosure.

Referring to FIGS. 11A to 11C, while a terminal image 1110 representingthe terminal 100 and sound source images 1111 to 1115 representing oneor more sound sources around the terminal 100 are being displayed on ascreen, upon receipt of a touch and hold input to the terminal image1110 as illustrated in FIG. 11A and then a flick input corresponding toa specific distance (e.g., 5 m) as illustrated in FIG. 11B, the terminal100 determines a sound source-selection distance according to thereceived inputs. As illustrated in FIG. 11C, the terminal 100 displays aradius corresponding to the sound source-selection distance from theterminal 100. Upon receipt of a hold input maintained for apredetermined time on the circumference of a circle having the radiusindicating the sound source-selection distance, successively to theflick input as illustrated in FIG. 11C, the terminal 100 determines theradius as the sound source-selection distance.

If the terminal 100 determines that the user gesture input indicates asound-source distance at operation 840, then once the soundsource-selection distance is determined, the terminal 100 proceeds tooperation 842 at which the terminal 100 may select a sound source withinthe sound source-selection distance and output a sound of the selectedsound source.

In contrast, if the terminal 100 determines that the user gesture inputdoes not indicate a sound-source distance at operation 840, then theterminal 100 proceeds to operation 850 at which the terminal 100determines whether the user gesture input indicates a soundsource-selection position. For example, a user gesture input indicatinga sound source-selection position may be a hold input at a specificpoint on the circumference of a circle having a radius defined by aflick input following a touch and hold input.

FIGS. 12A to 12D illustrate user gesture inputs indicating a soundsource-selection position according to an embodiment of the presentdisclosure.

Referring to FIGS. 12A to 12D, while a terminal image 1140 representingthe terminal 100 and sound source images 1141, 1143, 1145, 1147, and1147 representing one or more sound sources around the terminal 100 arebeing displayed on a screen, upon receipt of a touch and hold input tothe terminal image 1140 as illustrated in FIG. 12A and then a flickinput corresponding to a specific distance (e.g., 5 m) as illustrated inFIG. 12B, the terminal 100 may display a radius corresponding to thesound source-selection distance from the terminal image 1140 asillustrated FIG. 12C. Upon receipt of a hold input maintained for apredetermined time on the circumference of a circle having the radiusindicating the distance from the terminal 100, successively to the flickinput as illustrated in FIG. 12D, the terminal 100 may determine aposition on the circumference of the circle having the radius as a soundsource-selection position.

Once the terminal 100 determines the sound source-selection position atoperation 850, the terminal 100 proceeds to operation 852 at which theterminal 100 may select a sound source at the sound source-selectionposition and output a sound of the selected sound source.

According to various embodiments of the present disclosure as describedabove, the terminal 100 distinguishes adjacent sound sources around theterminal 100 and displays the sound sources separately. The terminal 100may output a sound of a user-selected sound source or control the volumeof the output sound.

The terminal 100 may receive sound signals of adjacent sound sourcesfrom the hearing aid 300 and display the sound sources. Upon userselection of a sound source, the terminal 100 may transmit a sound ofthe selected sound source to the hearing aid 300 so that the hearing aid300 may output the sound of the selected sound source.

If a user selects a volume level for a sound of a sound source, then theterminal 100 transmits the sound of the sound source with a volumecorresponding to the selected volume level to the hearing aid 300 sothat the hearing aid 300 may output the volume-controlled sound of thesound source.

A description will be given of the configuration and operation of thehearing aid 300 according to an embodiment of the present disclosure.

FIG. 13 is a block diagram of a hearing aid according to an embodimentof the present disclosure.

Referring to FIG. 13, the hearing aid 300 may include a hearing aidcontroller 310, a microphone unit 320, a converter 330, an amplifier340, an output unit 350, a wireless unit 360, a memory 370, and a powersupply 380.

The hearing aid controller 310 provides overall control to the hearingaid 300 and controls the operations of the microphone unit 320, theconverter 330, the amplifier 340, the output unit 350, the wireless unit360, the memory 370, and the power supply 380.

The microphone unit 320 may include at least two microphones and maydetect and receive adjacent sounds. For example, the at least twomicrophones may be disposed outside the hearing aid 300, receive analogsound signals from adjacent sound sources, in an ear of a user, andtransmit the analog sound signals to the converter 330 under the controlof the hearing aid controller 310.

The converter 330 may convert the analog sound signals received from themicrophone unit 320 to digital sound signals and may convert amplifieddigital sound signals to analog sound signals. For example, theconverter 330 may convert an analog sound signal received from themicrophone unit 320 to a digital sound signal and provide the digitalsound signal to the hearing aid controller 310.

The hearing aid controller 310 performs digital signal processes on thedigital sound signal received from the converter 330, such asunnecessary noise cancellation, feedback control, amplification gaincontrol, non-linear amplification, and the like. Thereafter, the hearingaid controller 310 provides the processed digital sound signal to theconverter 330.

The amplifier 340 amplifies the digital sound signal received from theconverter 330 based on an amplification gain received from the hearingair controller 310 and provides the amplified sound signal to thehearing aid controller 310.

The converter 330 converts the processed digital sound signal receivedfrom the hearing aid controller 310 to an analog sound signal andprovides the analog sound signal to the output unit 350.

The output unit 350 outputs the received analog sound signal.

The wireless unit 360 may include one of a mobile communication module(not shown), a WLAN module (not shown), a short-range communicationmodule (not shown), and the like, and may communicate with the terminal100 wirelessly. The short-range communication module may include aninfrared communication module, a Bluetooth communication module, and/orthe like. The wireless unit 360 communicates wirelessly with theterminal 100 under the control of the hearing aid controller 310. Thewireless unit 360 receives a sound signal of a selected sound source ora volume-controlled sound signal of the selected sound source from theterminal 100, or transmits sound signals of sound sources collectedthrough the microphone unit 320 to the terminal 100.

The memory 370 stores a sound signal of a selected sound source or avolume-controlled sound signal of the selected sound source, receivedfrom the terminal 100, or stores sound signals of sound sourcescollected through the microphone unit 320.

The power supply 380 supplies power to each component of the hearing aid300.

According to various embodiments of the present disclosure, the hearingaid 300 configured as described above may receive sound signals of soundsources through the microphone unit 320 and transmit the received soundsignals to the terminal 100. In addition, the hearing aid 300 mayreceive a sound signal of a user-selected sound source or avolume-controlled sound signal of the user-selected sound source fromamong sound signals of sound sources from the terminal 100 and outputthe received sound signal.

FIG. 14 is a flowchart illustrating an operation of a hearing aidaccording to an embodiment of the present disclosure.

Referring to FIG. 14, at operation 1410, the hearing aid 300 may receivesound signals of adjacent sound sources through the microphone unit 320.Specifically, the hearing aid 300 may receive analog sound signals fromadjacent sound sources through at least two microphones that may bedisposed outside the hearing aid 300 in an ear of the user.

At operation 1420, the hearing aid 300 may transmit the received soundsignals of the sound sources to the terminal 100. For example, thehearing aid 300 may communicate with the terminal 100 wirelessly throughthe wireless unit 360 and transmit the sound signals of the soundsources to the terminal 100 by wireless communication. The terminal 100may receive the sound signals of the sound sources from the hearing aid300, distinguish one or more sound sources based on the received soundsignals of the sound sources, calculate the sound strength, direction,and distance of each sound source, and display the one or more soundsources on a screen of the terminal 100.

Upon user selection of a sound source from among the displayed soundsources, the terminal 100 may transmit a sound signal of the selectedsound source to the hearing aid 300. Upon user selection of a volumelevel for the sound signal of the sound source, the terminal 100 maytransmit the volume-controlled sound signal of the sound source to thehearing aid 300. According to various embodiments of the presentdisclosure, the terminal 100 may transmit a sound signal of the selectedsound source to the hearing aid 300, and the terminal 100 may transmitan indication as to the selected volume level sound signal of theselected sound source.

At operation 1430, the hearing aid 300 may receive a sound signal orvolume-controlled sound signal of a selected sound source from theterminal 100 through the wireless unit 360.

At operation 1440, the hearing aid 300 outputs the received sound signalor volume-controlled sound signal of the selected sound source to theoutput unit 350.

While it has been described above that the hearing aid 300 transmitssound signals of sound sources received through the microphone unit 320to the terminal 100, the hearing aid 300 may receive from the terminal100 a sound signal or volume-controlled sound signal of a selected soundsource that the terminal 100 has received through the first to nthmicrophones 162-1 to 162-n, and then output the received sound signal,instead of transmitting sound signals of sound sources received throughthe microphone unit 320 to the terminal 100 through the wireless unit360.

As is apparent from the above description of the various embodiments ofthe present disclosure, a user can locate sound sources around the userusing the display of the terminal 100.

According to various embodiments of the present disclosure, the user may(selectively) hear only the sound of an intended sound source selectedfrom among the sound sources and control the volume of the soundstepwise.

According to various embodiments of the present disclosure, if a personsuffering from poor hearing uses the hearing aid 300, the person canlocate sound sources through the terminal 100 that operates inconjunction with the hearing aid 300 and hear the sound of an intendedsound source through the hearing aid 300.

Because the user can hear the sound of an intended sound source byvarious gestures, a convenient UI is provided to the user.

At this point it should be noted that the various embodiments of thepresent disclosure as described above typically involve the processingof input data and the generation of output data to some extent. Thisinput data processing and output data generation may be implemented inhardware or software in combination with hardware. For example, specificelectronic components may be employed in a mobile device or similar orrelated circuitry for implementing the functions associated with thevarious embodiments of the present disclosure as described above.Alternatively, one or more processors operating in accordance withstored instructions may implement the functions associated with thevarious embodiments of the present disclosure as described above. Ifsuch is the case, it is within the scope of the present disclosure thatsuch instructions may be stored on one or more non-transitory processorreadable mediums. Examples of the non-transitory processor readablemediums include Read-Only Memory (ROM), Random-Access Memory (RAM),CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.The non-transitory processor readable mediums can also be distributedover network coupled computer systems so that the instructions arestored and executed in a distributed fashion. Also, functional computerprograms, instructions, and instruction segments for accomplishing thepresent disclosure can be easily construed by programmers skilled in theart to which the present disclosure pertains

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A portable communication device comprising: atouchscreen display; memory to store audio data and position data, theaudio data including a first sound signal corresponding to a firstobject external to the portable communication device and a second soundsignal corresponding to a second object external to the portablecommunication device, the position data including first positioninformation corresponding to the first object and second positioninformation corresponding to the second object; and a processor adaptedto: present, via the touchscreen display, a plurality of indicationsincluding a first indication indicating the first sound signal at afirst position of the touchscreen display corresponding to the firstposition information, a second indication indicating the second soundsignal at a second position of the touchscreen display corresponding tothe second position information, and a third indication indicating theportable communication device at a third position of the touchscreendisplay, receive a user input with respect to the first indication orthe second indication, and reproduce a corresponding sound signal of thefirst sound signal and the second sound signal based at least in part onthe user input, the reproducing including adjusting a volume of thecorresponding sound signal based at least in part on the user input. 2.The portable communication device of claim 1, wherein the processor isadapted to: receive, via one or more microphones operatively coupledwith the portable communication device, the first sound signal from thefirst object, and the second sound signal from the second object.
 3. Theportable communication device of claim 1, wherein the processor isadapted to: determine a distance or a direction between the portablecommunication device with a corresponding one of the first object andthe second object based at least in part on the first sound signal orthe second sound signal; and store, in the memory, the distance or thedirection as at least part of a corresponding one of the first positioninformation and the second position information.
 4. The portablecommunication device of claim 1, wherein the processor is adapted to:mute, as at least part of the adjusting, the volume of the correspondingsound signal in response to the user input.
 5. The portablecommunication device of claim 4, wherein the processor is adapted to:receive another user input with respect to the selected indication; andoutput, via the speaker, the corresponding one of the first sound signaland the second sound signal based at least in part on the other userinput.
 6. The portable communication device of claim 1, wherein theprocessor is adapted to: adjust the volume of the corresponding soundsignal to a different volume as compared to another sound signal basedat least in part on the user input.
 7. The portable communication deviceof claim 1, wherein the processor is adapted to: change a color of acorresponding one of the first indication and the second indicationbased at least in part on the user input.
 8. The portable communicationdevice of claim 1, wherein the processor is adapted to: display thethird indication at a center in relation with the plurality ofindications, as at least part of the third position of the touchscreendisplay.
 9. The portable communication device of claim 1, wherein theprocessor is adapted to: receive another user input with respect to thethird indication; and adjust at least one volume of the first soundsignal or the second sound signal based at least in part on the otheruser input.
 10. The portable communication device of claim 9, whereinthe processor is adapted to: identify a touch input as at least part ofthe user input; and identify a drag input as at least part of the otheruser input.
 11. The portable communication device of claim 1, wherein atleast one of the first indication or the second indication comprisesdistance information, an azimuth angle, a type of a corresponding one ofthe first object and the second object, sound strength of acorresponding one of the first sound signal and the second sound signal,or any combination thereof.
 12. The portable communication device ofclaim 1, wherein at least one of a size, a type, or a name of the firstindication is different from a corresponding one of a size, a type, or aname of the second indication based at least in part on the firstposition information and the second position information.
 13. A portablecommunication device comprising: a touchscreen display; and a processoradapted to: present, via the touchscreen display, a first indicationindicating a first sound signal at a first position of the touchscreendisplay corresponding to a first direction, a second indicationindicating a second sound signal at a second position of the touchscreendisplay corresponding to a second direction. output, via a speakeroperatively coupled with the portable communication device, the firstsound signal and the second sound signal; select an indication from thefirst indication and the second indication based at least in part on auser input, and stop the outputting of a corresponding sound signal ofthe first sound signal and the second sound signal corresponding to theselected indication.
 14. The portable communication device of claim 13,further comprising: a plurality of microphones, wherein the processor isadapted to: receive, via the plurality of microphones, the first soundsignal and the second sound signal.
 15. The portable communicationdevice of claim 14, wherein the processor is adapted to: detect thefirst direction and the second direction using the plurality ofmicrophones.
 16. The portable communication device of claim 13, whereinthe processor is adapted to: present a third indication indicating theportable communication device at a third position of the touchscreendisplay.
 17. The portable communication device of claim 16, wherein theprocessor is adapted to: receive another user input with respect to thethird indication; and adjust at least one volume of the first soundsignal or the second sound signal based at least in part on the otheruser input.
 18. A portable communication device comprising: atouchscreen display; one or more microphones to receive a first soundsignal from a first object external to the portable communication deviceand a second sound signal from a second object external to the portablecommunication device; and a processor adapted to: present a firstindication indicating the first sound signal at a first position of thetouchscreen display corresponding to the first object, and a secondindication indicating the second sound signal at a second position ofthe touchscreen display corresponding to the second object, receive, viathe touchscreen display, a user input with respect to the firstindication or the second indication, and adjust a volume of acorresponding sound signal of the first sound signal and the secondsound signal based at least part on the user input.
 19. The portablecommunication device of claim 18, wherein the processor is adapted to:select an indication from the first indication and the second indicationbased at least in part on the user input; and output, via a speakeroperatively coupled with the portable communication device, a soundsignal of the first sound signal or the second sound signalcorresponding to the selected indication.
 20. The portable communicationdevice of claim 18, wherein the processor is adapted to: change agraphic user interface of a corresponding one of the first indicationand the second indication based at least in part on the user input.